Integration of multimodal data based on surface registration

The paper proposes and evaluates a strategy for the alignment of anatomical and functional data of the brain. The method takes as an input two different sets of images of a same patient: MR data and SPECT. It proceeds in four steps: first, it constructs two voxel models from the two image sets; next, it extracts from the two voxel models the surfaces of regions of interest; in the third step, the surfaces are interactively aligned by corresponding pairs; finally a unique volume model is constructed by selectively applying the geometrical transformations associated to the regions and weighting their contributions. The main advantages of this strategy are (i) that it can be applied retrospectively, (ii) that it is tri-dimensional, and (iii) that it is local. Its main disadvantage with regard to previously published methods it that it requires the extraction of surfaces. However, this step is often required for other stages of the multimodal analysis such as the visualization and therefore its cost can be accounted in the global cost of the process.Postprint (published version

A Fast hierarchical traversal strategy for multimodal visualization

In the last years there is a growing demand of multimodal medical rendering systems able to visualize simultaneously data coming from different sources. This paper addresses the Direct Volume Rendering (DVR) of aligned multimodal data in medical applications. Specifically, it proposes a hierarchical representation of the multimodal data set based on the construction of a Fusion Decision Tree (FDT) that, together with a run-length encoding of the non-empty data, provides means of efficiently accessing to the data. Three different implementations of these structures are proposed. The simulations results show that the traversal of the data is fast and that the method is suitable when interactive modifications of the fusion parameters are required.Postprint (published version

Can 3D gamified simulations be valid vocational training tools for persons with intellectual disability? A pilot based on a real-life situation

Objective: To investigate if 3D gamified simulations can be valid vocational training tools for persons with intellectual disability. Methods: A 3D gamified simulation composed by a set of training tasks for cleaning in hostelry was developed in collaboration with professionals of a real hostel and pedagogues of a special needs school. The learning objectives focus on the acquisition of vocabulary skills, work procedures, social abilities and risk prevention. Several accessibility features were developed to make the tasks easy to do from a technological point-of-view. A pilot experiment was conducted to test the pedagogical efficacy of this tool on intellectually disabled workers and students. Results: User scores in the gamified simulation follow a curve of increasing progression. When confronted with reality, they recognized the scenario and tried to reproduce what they had learned in the simulation. Finally, they were interested in the tool, they showed a strong feeling of immersion and engagement, and they reported having fun. Conclusions: On the basis of this experiment we believe that 3D gamified simulations can be efficient tools to train social and professional skills of persons with intellectual disabilities contributing thus to foster their social inclusion through work.Postprint (author's final draft

Speeding up rendering of hybrid surface and volume models

Hybrid rendering of volume and polygonal model is an interesting feature of visualization systems, since it helps users to better understand the relationships between internal structures of the volume and fitted surfaces as well as external surfaces. Most of the existing bibliography focuses at the problem of correctly integrating in depth both types of information. The rendering method proposed in this paper is built on these previous results. It is aimed at solving a different problem: how to efficiently access to selected information of a hybrid model. We propose to construct a decision tree (the Rendering Decision Tree), which together with an auxiliary run-length representation of the model avoids visiting unselected surfaces and internal regions during a traversal of the model.Postprint (published version

Rendering techniques for multimodal data

Many different direct volume rendering methods have been developed to visualize 3D scalar fields on uniform rectilinear grids. However, little work has been done on rendering simultaneously various properties of the same 3D region measured with different registration devices or at different instants of time. The demand for this type of visualization is rapidly increasing in scientific applications such as medicine in which the visual integration of multiple modalities allows a better comprehension of the anatomy and a perception of its relationships with activity. This paper presents different strategies of Direct Multimodal Volume Rendering (DMVR). It is restricted to voxel models with a known 3D rigid alignment transformation. The paper evaluates at which steps of the render-ing pipeline must the data fusion be realized in order to accomplish the desired visual integration and to provide fast re-renders when some fusion parameters are modified. In addition, it analyzes how existing monomodal visualization al-gorithms can be extended to multiple datasets and it compares their efficiency and their computational cost.Postprint (published version

Design of a multimodal rendering system

This paper addresses the rendering of aligned regular multimodal datasets. It presents a general framework of multimodal data fusion that includes several data merging methods. We also analyze the requirements of a rendering system able to provide these different fusion methods. On the basis of these requirements, we propose a novel design for a multimodal rendering system. The design has been implemented and proved showing to be efficient and flexible.Postprint (published version

Hybrid model for vascular tree structures

This paper proposes a new representation scheme of the cerebral blood vessels. This model provides information on the semantics of the vascular structure: the topological relationships between vessels and the labeling of vascular accidents such as aneurysms and stenoses. In addition, the model keeps information of the inner surface geometry as well as of the vascular map volume properties, i.e. the tissue density, the blood flow velocity and the vessel wall elasticity. The model can be constructed automatically in a pre-process from a set of segmented MRA images. Its memory requirements are optimized on the basis of the sparseness of the vascular structure. It allows fast queries and efficient traversals and navigations. The visualizations of the vessel surface can be performed at different levels of detail. The direct rendering of the volume is fast because the model provides a natural way to skip over empty data. The paper analyzes the memory requirements of the model along with the costs of the most important operations on it.Postprint (published version

Frame-to-frame coherent image-aligned sheet-buffered splatting

Splatting is a classical volume rendering technique that has recently gained in popularity for the visualization of point-based suface models. Up to now, there has been few publications on its adaptation to time-varying data. In this paper, we propose a novel frame-to-frame coherent view-aligned sheet-buffer splatting of time-varying data, that tries to reduce as much as possible the memory load and the rendering computations taking into account the similarity in the data and in the images at successive instants of time. The results presented in the paper are encouraging and show that the proposed technique may be useful to explore data through time.Postprint (published version

Navegación virtual por un escenario teatral

The paper describes the design of a post-wimp interface for an application of virtual navigation in a theater. The application is aimed at stimulating the interest of young children for theater in general and opera in particular. Using the navigator before and after a concert, the theatral experience is extended throughout time. The interface is designed taking into account that users may not have computer parctice and even may not know reading. Therefore, the classical menus have been removed from the interface and substituted by graphical objects with a reduced number of functions that simulate the actual behaviour of the entities that they represent. The application is implemented in VRML and can be launched from internet. The usability tests show that it is easy to learn and use without need of written documentation.Postprint (published version

First-person locomotion in 3D virtual environments: a usability analysis

3D Virtual Environments (VE) are becoming popular as a tool for cognitive, functional and psychological assessment. Navigation in these environments is recognized as one of the most difficult activities in 3D Virtual Environments (VE). Users unfamiliar to 3D games, specially elder persons, get puzzled when they try to virtually move an avatar through these environments. Their inability to navigate prevents them from concentrating in the task and even to finish it. In this paper, we analyze the influence of different factors in locomotion control. We investigate the impact of having the cursor fixed at the camera center or leaving it free inside the current view. We also analyze the influence of the pitch angle on the camera control. In addition, we have designed an automatic locomotion system that we compare to user-controlled locomotion. We describe a virtual scenario and a test task that we have implemented to evaluate these different methods with users of diverse profiles.Postprint (published version